1. Field of the Invention
The present invention relates to the technology of an imaging apparatus and an image generating method, and more specifically to the technology of an imaging apparatus and an image generating method for a fluorescent sample.
2. Description of the Related Art
There is an imaging apparatus well known for converting the light from a sample into an electric signal, and generating an image of the sample.
The imaging apparatus has an imaging unit having a solid-state image pickup device for performing an optoelectric conversion. The imaging unit normally has a plurality of pixel units capable of independently detecting the quantity of light. When an image is generated using the imaging unit with the configuration above, the resolution of the image is limited by not only the resolution of an optical system (hereinafter referred to as an optical resolution) between the sample and the imaging unit, but also the number of pixel units (hereinafter referred to as the number of pixels) included in the imaging unit.
Accordingly, although the optical resolution of an optical system is sufficiently high, the imaging unit limits the resolution of the image if the number of pixels is small and the resolution which can be realized is low, thereby failing in acquiring an image of a satisfactory resolution.
For example, FIG. 1 exemplifies an imaging apparatus having a microscope between the sample and the imaging unit. In an imaging apparatus 400 exemplified in FIG. 1, an imaging apparatus body 200 such as a digital camera for a microscope is connected to a port 101 of a microscope 100, and the imaging apparatus body 200 captures a sample image output from the port 101.
For more details, the light emitted from the light source 102 provided for the microscope 100 is converted into parallel light through a collector lens 103, reflected by a mirror 104, and irradiated to a sample 109 through a windows lens 105, field stop 106, aperture stop 107 and a condenser lens 108. The light passing through the sample 109 is output as a sample image from the port 101 through an objective 110 and a tube lens 111.
The imaging apparatus body 200 is provided with an image pickup device 201 such as a CCD functioning as an imaging unit for capturing a sample optical image output from the port 101 of the microscope 100. The image pickup device 201 is driven at an exposing time according to a drive signal from an image pickup device drive unit 202, and inputs an output signal to a preprocessing unit 203. The preprocessing unit 203 converts the output signal of the image pickup device 201 into a video signal according to a control pulse provided from the image pickup device drive unit 202, and inputs the resultant signal to an A/D conversion unit 204. The A/D conversion unit 204 outputs the video signal as digital image data to a signal processing unit 205 according to the clock signal from the image pickup device drive unit 202. Afterwards, the image data is processed for a signal by the signal processing unit 205 performing a color correction, a gray-scale correction, etc., converted into an analog signal by a D/A conversion unit 206, and displayed as moving pictures on a display unit 207.
In addition, a control unit 209 which has received an instruction from an operation unit connected to an I/F unit 208 transmits the image data from the signal processing unit 205 to a record unit 211 through a bus 210, and records the data as a still image.
FIG. 2 exemplifies an imaging apparatus having a microscope similar to that illustrated in FIG. 1 between the sample and the imaging unit. A imaging apparatus 401 exemplified in FIG. 2 includes a personal computer (PC) 300 in addition to the microscope 100 and the imaging apparatus body 200. In the imaging apparatus 401, the D/A conversion unit 206, the display unit 207, and the record unit 211 are deleted, and are replaced with the PC 300 connected to an I/F unit 212 and having the functions of the deleted units. Apart of image processing can also be performed by the PC 300.
In the imaging apparatuses 400 and 401 exemplified respectively in FIGS. 1 and 2, when the number of pixels of the image pickup device 201 is small, the optical properties of the microscope 100 cannot fully work although the optical resolution of the microscope 100 is sufficiently high, thereby failing in obtaining an image of a satisfactory resolution.
To solve the above-mentioned problem, there is a countermeasure for improving the resolution by increasing the number of pixels of the image pickup device 201 (imaging unit). However, in this case, it is pointed out that there occurs a new problem with the imaging unit about a production cost, an optoelectric conversion efficiency, etc.
On the other hand, there is a pixel shifting technique proposed as technology for improving the resolution of an image generated by an imaging apparatus without increasing the number of pixels of the imaging unit. The pixel shifting technique improves the resolution of an image up to or exceeding the resolution of the imaging unit by combining a plurality of images having different relative positions of the imaging unit for a sample optical image by image processing. It is disclosed by, for example, Japanese Laid-open Patent Publication No. 8-251604 etc.
FIG. 3 exemplifies the configuration of the imaging apparatus using the pixel shifting technique. An imaging apparatus 402 exemplified in FIG. 3 is different from the imaging apparatus 401 exemplified in FIG. 2 in that the imaging apparatus 402 has a device shift unit 213 (pixel shift unit) for moving the image pickup device 201. The imaging apparatus 402 acquires images at different relative positions by changing the relative position of the image pickup device 201 for a sample image using the device shift unit 213. Then, by combining the acquired images, an image having a resolution equal to or exceeding the resolution of the image pickup device 201 can be generated.
For example, as exemplified in FIG. 4A, if the imaging apparatus by the pixel shifting technique captures an optical image of a fluorescent sample 500 having a cell area 601 including areas 602 and 603 dyed by the respective fluorescent coloring agents and a background area 604 including no fluorescent coloring agent, then an image 510 exemplified in FIG. 4B is generated.
Described below with reference to FIGS. 5A through 5C and 6A through 6I is the reason for an occurrence of a checkered pattern as exemplified in FIG. 4B in the fluorescent image generated by the imaging apparatus.
The imaging apparatus by the pixel shifting technique acquires a plurality of images having different relative positions of the imaging unit for a sample optical image.
Practically, for example, the imaging unit is configured so that the pixel unit of red (R) can be arranged at an initial pixel position 701, the pixel unit of green (G) can be arranged at initial pixel positions 702 and 703, and the pixel unit of blue (B) can be arranged at an initial pixel position 704 as exemplified in FIG. 5B in the state in which the imaging unit is at a first relative position Pos 1 exemplified in FIG. 5A. In addition, the relative position of the imaging unit is sequentially changed from the first relative position Pos 1 to a ninth relative position Pos 9 so that the pixel shift unit can make a shift by ⅔ pixel pitch for each relative position. Then, an image of the sample 500 is acquired at each relative position. Thus, a bayer pattern 700 exemplified in FIG. 5B is generated.
An image 501 exemplified in FIG. 6A through an image 509 exemplified in FIG. 6I are exemplified as an image of the sample 500 acquired when the imaging unit is positioned respectively at the first relative position Pos 1 through the ninth relative position Pos 9.
As exemplified in FIGS. 6A through 6I, with the images (images 501 through 509) of the sample 500, the brightness of the cell area 601 decreases with the progress of the acquisition of images, thereby obtaining a darker image. Since there occurs a time difference between the acquisition times of images, time passes as images are acquired, and the fluorescent coloring agent for dyeing the cell area 601 fades, thereby decreasing the fluorescence generated by a fluorescent coloring agent. Therefore, as with a bayer pattern 710 exemplified in FIG. 5C, the quantity of light entering each adjacent pixel unit aiming at the same portion of the cell area 601 are also different depending on the acquisition time by the influence of the fading. In FIG. 5C, the quantity of light entering each pixel unit is expressed by a gray-scale level of each pixel unit.
Thus, when the imaging apparatus by the pixel shifting technique is used in a fluorescence observation, acquired are a plurality of images having different brightness levels depending on the fading of the fluorescent coloring agent. Since an image is generated by combining the images at different brightness levels, a checkered pattern occurs on the generated image.
As exemplified in FIG. 4B, the checkered pattern occurs in the cell area 601 dyed by a fluorescent coloring agent. Therefore, to remove the checkered pattern, there is a method proposed to adjust the exposing time for each of the images different in relative position by considering the fading of the fluorescent coloring agent. In this method, the brightness of the background area 604 is different for each image, and the checkered pattern occurs in the background area 604 unlike the case illustrated in FIG. 4B.